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A number of problems are considered relevant to understanding the 
performance of optical-fiber communication systems that use pulse 
transmission. The methods used are typically exact solutions or bounds, 
and we concentrate on simple examples that aid our understanding. 
Some of our work makes contact with previous studies, particularly by 
Personick and Hubbard. The major results are: 

(i) Presentation of an integral equation for the output density for single- 
pulse detection with arbitrary avalanche gain 
(ii) Exact solution for the probability distribution for gains in physical 

avalanche diodes 
(Hi) Bounds on performance when intersymbol interference is present 
(but 110 avalanche gain) which suggest that an optimum-bit de- 
tector can perform, under practical conditions, only two or three 
dB better than a simple integrate-and-dump filter, yielding results 
still many dB from the quantum limit. Thus, in particular, little 
performance gain is to be expected from equalization techniques. 

I. INTRODUCTION AND OVERVIEW 

A large part of traditional communication theory has been directed 
to detecting and processing electrical signals transmitted over wires, 
cables, or the like. While the physical realization of each of these tradi- 
tional systems may have led to mathematical treatments designed to 
handle problems such as linear distortion or fading, which were peculiar 
to one, or even perhaps several, systems, the principal concern of all 
mathematical treatments of these time-continuous channels has been 
the ubiquitous additive gaussian noise. In fact, it would be fair to 
say that much of the structure of the mathematical treatments used 
has been dictated by the mathematical properties of this noise. In the 
absence of noise, many problems would immediately degenerate, at 
least theoretically, to situations of perfect detection, infinite capacity, 
etc. 
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The consideration of some promising optical communication systems 
seems to alter the above picture. We have in mind the transmission of 
information by way of light pulses propagating through an optical 
fiber and subsequently detected by a photodetector that converts 
electromagnetic energy in the fiber to electrical signals in a circuit. 
We immediately note certain features which this problem has in common 
with the traditional problems. For one thing, the fiber can delay, 
attenuate, or spread the transmitted pulses. For another, the elec- 
trical signal after photodetection may be corrupted by additive 
gaussian noise. Yet there is another fundamental impairment. The elec- 
tromagnetic signal that propagates in the fiber (which acts as a wave 
guide) is, under practical considerations, of sufficiently weak intensity 
that any effective detection mechanism must be based upon the 
quantum nature of the electromagnetic disturbance. In other words, 
detection must be based upon photon counting. Here, a new element 
enters the problem — photon counting is subject to statistical fluctua- 
tions. In the quantum case, a signal uncorrupted by any external 
disturbance still carries with it its own "noise," as it were, which is 
not additive gaussian. This new noise manifests itself in the following 
way. The photon-counting process is a time-varying Poisson process 
whose intensity (or rate) function \(t) varies in direct proportion to the 
information-bearing pulse train, the latter being thought of in the 
conventional way (except it must now always be positive). Our pur- 
pose here is to explore some of the communication theory of this new 
situation, paying particular attention to the use of our considerations 
in proposed fiber-optic communication systems. 

The general background of the material that we treat, namely, 
direct detection of photons in an optical fiber, may be found in works 
by Personick 1 - 2 and Foschini et al. 3 Direct detection refers to the pro- 
cessing of the electrical signal at the output of a photodetector as op- 
posed to, say, more esoteric detection schemes based on optimum 
processing of the existing electromagnetic field, considered as a 
quantum system. In the case of binary transmission, the choice be- 
tween a one or a zero is, in the systems considered here, translated into 
the presence or absence of a short burst of optical power (light) in the 
fiber. To understand this in more detail, we shall trace the passage of 
a single pulse through our mathematical model of the system (see 
Fig. 1). In the case of a one being transmitted, an electrical signal (a 
square pulse of duration T) turns on our "flashlight," which in this 
case is a laser or light-emitting diode, and electromagnetic energy is 
sent into the transmission medium (optical fiber). If a photodetector 
is placed at the end of the fiber, photons will be detected due to the 
electromagnetic energy present. Exactly when in time the photons 
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Fig. 1 — Passage of a single pulse through the optical system. 



register on the detector is random and is the Poisson process spoken 
of earlier. The probability of receiving a count between time t and 
t + dt is given by h(t)dt, where, owing to effects in the fiber, h(t) is a 
distorted and attenuated version of the transmitted pulse. The ac- 
cumulation of distortion as the pulse propagates down the fiber is also 
sketched in Fig. 1. In practice, a background of counts also exists. 
This is called the dark current and is modeled by introducing a con- 
stant additive intensity function X before the detector, although some 
of these counts can originate in the physical detector itself. Typically, 
transmitted power and transmission loss are adjusted so that on the 
order of one or two hundred photons per pulse are, on the average, 
detected. The dark current contributes from about 1 to 5 percent of 
the counts. 

To transmit a zero, we simply do not turn on the transmitting power, 
and the detector only registers counts resulting from the dark current. 

We have been loosely speaking of the output of the photodetector 
as "counts." The actual electrical current at the output of this device 
caused by a photon is a wideband pulse g-w(t) (very narrow com- 
pared with T, a delta function in the limit), where g = integer- valued 
random variable or g = 1, depending on whether or not an avalanche 
diode is used. The electrical current at the output of the photodetector 
is further distorted by gaussian noise whose effect is often lessened in 
importance when an avalanche diode is used, but not for the g = 1 
case. In the most literal modeling of the experimental situation, the 
finite bandwidth of w (t) prevents one from assuming that the Poisson 
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part of the observation is singular, i.e., can be separated out from the 
background gaussian noise; however, whenever we feel there are 
insights to be gained from the separation we shall make it. 

If we take into account the facts that Personick 4 has shown super- 
position to hold (approximately) for optical-fiber transmission and 
that optical power is positive, then we may extend our single-pulse 
description to a model for transmission of an entire pulse train. If we 
transmit a sequence of on or off pulses, then the "received signal," 
defined as that electrical signal on which we may do processing, can 
be written as 

L g n w(t - t n ) + n(t), (1) 

n 

where the time points {t n \ form a Poisson process having intensity 
function \(t), with 

X(0 = £ a n h(t - nT) + X„ (2) 

n 

and 

h (t) ^ = distorted pulse 
a„ = 0, 1 = independent, equiprobable data symbols 
X ^ = dark current 

T = signaling interval (3) 

n(t) = gaussian noise 

g„ = avalanche gain factors 
w(t) = output pulse of photodetector. 

At various stages of our discussion, we may, for interests of simplicity 
or clarity, idealize or eliminate certain aspects of the full model given 
by (1), (2), and (3). 

The communication theorist is interested in processing the signal 
(1) to estimate the a„ given in (2). If the distortion is not severe, one 
may simply process in an intuitive way and (assuming proper syn- 
chronization) count the number of photons detected in the appropriate 
T-second interval. If g» «= 1, this is accomplished by integrating the 
output for T seconds (so-called integrate-and-dump detection). How- 
ever, the simplicity of this technique demands its investigation even 
when g n are random. Neglecting the gaussian noise and assuming g n 
are exponential random variables allow one to determine exactly the 
probability distribution of the output statistic and to determine error 
rates. This is done in Section II. In Section III we return to the g = 1 
case to observe the effects of the random gain. In Section IV, 
Personick's implicit equation for the random gains g n of actual photo- 
detectors is studied in detail and the exact distribution of these gains 
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is found. Also, the use of Chernoff bounds for bounding the error rate 
in the general situation is discussed. Section V branches out to include 
a worst-case analysis of intersymbol interference [the case of appre- 
ciable spreading of /t(0D using integrate-and-dump detection. A 
particular example is also computed. Finally, in Section VI, we con- 
sider the question of replacing the integrate-and-dump detector with 
an optimum detector. We know that equalization can achieve con- 
siderable improvements for voiceband telephone transmission, but 
can we expect the same here? Using the lower bound on performance 
which we derive for the optimum detector and applying this to the 
example of Section V, we find that performance greatly surpassing that 
of integrate-and-dump detection cannot be expected. 

II. INTEGRATE-AND-DUMP DETECTION— AVALANCHE DETECTORS 

As already mentioned in the introduction, a simple way to detect 
the jth bit in (2) is to integrate the output of the photodetector over 
the jth T-second interval and compare the random variable thus 
obtained with a threshold F ; if the output is greater than F, a one is 
declared (pulse present) ; if it is less than F, a zero is declared (pulse 
absent). In this section, we discuss the exact error rate for such a 
situation when pulse overlap in (2) can be neglected, as well as the 
additive noise. Further, the gains g n are assumed to be exponentially 
distributed. 

We shall need the moment-generating function (mgf) for the indi- 
cated random variable, but we may as well begin by giving the mgf 
for a general linear filter P(t) rather than simply an integrator. Con- 
sider a Poisson point process having an arbitrary intensity function 
X(0 [not necessarily of the form (2)], and let the nth count be given 
nonnegative weight g„, i.e., consider 

E gM - tn), (4) 

where the sequence of time points {t„} is Poisson with intensity func- 
tion \(t). If (4) is linearly filtered, with P(t) being the impulse response 
of the filter, then the output of the filter at time t, x(t) can be shown by 
elementary calculations to have moment-generating function given by 

M x = -Eexp [sXJ = exp \ T A(r){M fl [sP(* - r)] - l}drl (5) 

where M„(s) is the moment-generating function of the g n , assumed 
independent, and we have set x{l) = X. In particular, if P(r) = 1 for 
< t < T and zero elsewhere, and if t = T, (5) will simplify to 

M x = exp (A[Jf.(«) - 1]}, (6) 
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where 

A=f T \(r)dT. (7) 

If the pulse w(t) in (1) (assumed of unit area) is narrow enough so 
that end effects are negligible when doing the integration and if pulse 
overlap in (2) is negligible, then (6) and (7) are relevant quantities to 
consider in determining the error rate for integrate-and-dump detec- 
tion of (1) and (2). To treat the two separate cases of a one or a zero, 
we need only replace A in (6) by A,, i = 1 or 0, where 

Ai = C h{t) + T\ (8) 

Jo 

A = T\ . (9) 

While the gaussian noise will be neglected here, let us at least note 
that to include the effect of the added noise term on the integrated 
output, we would multiply (6) by the moment-generating function of 
the noise M n (s), 



M„(s) = exp 



(£), do 



to obtain the mgf of the new output variable. In (10), the variance of 
the noise a 2 is given by 

a* = ^ T (11) 

for the case of the integrator with white noise of two-sided spectral 
density No/2, or 

<r«-~ ^ N(a)\P(<*)\*du, (12) 

in general, where JV(«) denotes a general noise spectrum and P(co) 
is the Fourier transform of P(t). 

In the special case where the g„ in (4) are continuous variables and 
are exponentially distributed, i.e., 

p(g) = a exp (-ag), g > 0, (13) 

we have 

M g (s)=— — , s<a. (14) 

" a — s 

At this stage, it is easier to work with the characteristic function 
version of (6), namely, 

C x (o>) = exp {A[C,(«) - 1]}, (15) 

352 THE BELL SYSTEM TECHNICAL JOURNAL, MARCH 1976 



with C(o)) denoting characteristics functions now, e.g., 

Cx(w) = -Bexp (iaX). 

To obtain our integral equation for p(x), differentiate (15) once with 
respect to o>, multiply by exp ( — iux), and integrate over x to obtain 

xp{x) = A / up g (u)p(x — u)du, (16) 

Jo 

where p(x) denotes the density of the random variable X in (14), and 
p„(u) denotes the density of the nonnegative gain variable g. 

For the exponential gain case (13), an exact solution to (16) can be 
found. Note that then the variable x has probability exp ( — A of 
being zero (no counts) and p(x) will thus contain a 5 function at the 
origin. Introducing this explicitly by writing 

p(x) = exp (-A)5(x) + exp (-ax)f(x), (17) 

we find 

xf(x) = (aAe~ x )x + aA (* (x - w)f(w)dw, (18) 

Jo 

where use has been made of (13). Differentiating (18) twice, we obtain 
Bessel's equation 

x 2 /" + 2xf - (aA)xf = 0, (19) 

where /' stands for differentiation. The appropriate solution of (19) 
gives, finally, for the density p(x) of the detection statistic 

p(x) = e~ A 8(x) + e~ A J— e— *7i(2-^A«) > (20) 

/i(-) being the modified Bessel function.* The following may be useful 
in connection with (20) : 

h(x)^^M, x^O (21) 

■\2vx 

J^^^L^, x large (22) 

Ii(x)~%, a: small. (23) 

Typically, Ai is in the range of 100 to 200 for a light pulse present and 

* This exact result, as well as several useful approximations to it found later in this 
section, were first derived by Hubbard (lief. 5) using other techniques. 
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A in the range of 5 to 10 for dark current only. The quantity l/«, the 
average gain, may be 100 or 200. The average number of counts for a 
pulse is then Ai/a so, to within a factor of 2 or so, the decision threshold 
will be around Ai/2a. Thus, virtually all the area of interest in (20) 
occurs for x > l/aA, for both i = 1 or 2, and (22) may be used and, 
to excellent accuracy, 

p(x)dx „ ^ exp | 2(1/2a) | d *> x > aA (24) 



Equation (24) is slightly more attractive if we write instead the density 
for u = Vz, Pu(u), 

(u - 41/ a) 2 1 



/A\i 1 P l 2(l/2a J, ^ 1 /OK , 

p u {u)dutt - ) -;= .— , ,-Y / — - du, u > - T =, (25) 

\<x/ ^ V2^(\T/2a) \£X 

s howi ng that Vx is, over a rather wide range, gaussian with mean 
VA/a and variance l/2a. Note A/a = EZ, while variance of X is 
2A/a 2 . Also, eq. (25) should not be confused with the central limit 
theorem version of (24), which is obtained when one writes (for large 
A) x = (A/a) + e and e becomes gaussian. Since, from (21), eq. (22) 
is an upper bound as well as being asymptotic, we have 



p[x>F >^]^(AyQ ( V2^-V2A), 



(26) 



where 

Q(v) = -k= / e-^du~ e -j==- (27) 

Likewise, in the same spirit of approximation that indicates (26) to be 
an excellent approximation (in addition to being an upper bound) , one 
may write for the lower tail 



p\x < F < £ 1 tt ( ^ Vq(V2A - J2a~F). 



(28) 



Even for A's differing by a factor of 100, the fourth root factor in front 
of (26) and (28) is weak indeed. Thus, we may, to excellent approxima- 
tion, find the best threshold by equating the arguments of the Q func- 
tion for the two cases of error. This results in 

V2A - ! - VW = -j2a~F - V2A^. (29) 

The left-hand side of (29) comes, of course, from using (28) for a pulse 
present (the number of counts is then expected to exceed the threshold). 
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Table I — Tabulation of error rate and threshold for an avalanche 
detector with exponentially distributed gains 



Ao 


A„ 


F opi (<* = 1) 


P c [eq. (31)] 


Quantum Limit 


4 


100 


37.20 


2.09 X lO- 9 


1.86 X 10~ 44 


4 


200 


66.28 


8.88 X 10- 19 


~ 10- 88 


4 


400 


122.1 


3.9 X 10- 38 


~ lO-i" 


10 


100 


46.58 


8.03 X 10~ 8 


1.86 X 10- 44 


10 


200 


77.91 


3.61 X 10~ 16 


~ 10" 88 


10 


400 


137.0 


3.66 X 10~ 34 


~ 10-' 76 



Likewise, the right member of (29) comes from using (26) for only 
dark current where the counts usually fall below the threshold F and 
an error is made only if they exceed it. We immediately obtain from 
(29) 

(30) 

where, again, A is not to be too small, for example, Ao ^ 2. In the 
above, we have in mind, from (8), taking Ai = Ai, + A where Ai, is 
due to signal alone. 

For future comparisons, we should inject at this point the fact that 
the best detection probability one can obtain with no dark current 
(or no gaussian noise) is \ exp ( — Ai„), often referred to as the quantum 
limit. 

Table I displays values of the right member of (26), for the optimum 
F given by (30), i.e., it displays the quantity 



m<^-S) 



(31) 



evaluated for several values of A and A is , along with the quantum 
limit. Note that only aF enters the expressions, and thus the actual 
value of a plays no role in determining the probabilities for this prob- 
lem. The fact should also be evident from the scaling properties of 
the problem. In real applications, 1/a would be large so that the 
electronic circuitry could "see" the pulses above the gaussian noise. 
Table I shows (for the parameters shown) about a 7-dB loss rela- 
tive to the quantum limit, owing to the dark current, and also in part 
to the random nature of the gain mechanism.* 



* To be perfectly clear on this point, it is really the additional (random) gain 
provided by the avalanche detector that allows one to formulate the physical problem 
as in (4) without gaussian noise. However, from a mathematical point of view, once 
(4) is written down, the random gains are hypothesis-insensitive, and thus would be 
ignored by an optimum detector. 
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III. INTEGRATE-AND-DUMP DETECTION— PURE POISSON CASE 

We now give a brief discussion for the g = 1 case of (4), namely, 
the random variable X is Poisson, 

p(X = n) = ^p n = 0, 1,2, •••, (32) 

EX = A, var X = A 2 . (33) 

The purpose of the remarks will be to shed light on the degradation 
suffered when the g n are random, as mentioned at the end of the last 
section. 

If X is Poisson, then the probability that X is larger than or equal 
to k is 

£ k -^r ~ ~rr L 1 + )m + <* + d<* + 2) + J (34) 

If, in addition, we assume (k + 1) > A, then a simple consequence of 
(34) is that 

( 1 + .-TT)^t ! < Pr ^^> A - 1 ^ 

1 e~ A A* , . 

<1- (A/k + l)'~kT' ^ 6) 

Similarly, for the lower tail we. have 

K)^*^*^™^ (36) 

Thus, ignoring the weak effects of the coefficient in front, the 
optimum threshold F for a problem such as the one described in Sec- 
tion II is obtained by equating probabilities such as these in (35) and 

(36), yielding 

e-Ao A £ = e -A, A ^ (37) 

or, equivalently, the optimum threshold in this case is 

**- wa~/aV (38) 

In (Ai/Ao) 

Table II displays the right-hand side of (35) for k given by the 
rounded-off values of (38). In particular, we see degradation ranging 
from 3.5 to 4 dB compared to the quantum limits given in Table I. 
Typically, then, detecting the presence or absence of a single pulse 
using random amplitudes, as a linear detector might, results in a 3- to 
4-dB degradation (for the exponential case), compared with an "ideal" 
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Table II — Tabulation of error rate and threshold for detection 

with constant gain 



A 


Ai. 


^opt 


P. [eq. (35)] 


4 


100 


30.69 


1.17 X 10-" 


4 


200 


50.87 


6.49 X lO" 38 


4 


400 


86.67 


2.18 X 10~" 


10 


100 


41.70 


4.21 X 10~ M 


10 


200 


65.69 


9.80 X lO" 32 


10 


400 


107.7 


3.78 X 10-" 



avalanche detector, which has a large gain but whose distribution is 
concentrated at a delta function. 

The loss due to "gain jitter" suggests a possible remedy. The 
physical pulse g n w(t — t„) in the detection circuits following the 
avalanche diode should be clearly detectable against the background 
noise if g n is sufficiently large; in particular, if it is something like the 
mean gain G. Suppose this is also true for pulse gains g„ ^ fG, f < 1. 
Now suppose one processed the circuit output of the avalanche diode 
by first passing it through a pulse detector that detects pulses of height 
greater than fG and generates a pulse of fixed height if a pulse is 
detected. The output pulses of this device have fixed gain, which is 
beneficial, but, on the other hand, we have lost a fraction d, 



1 f fG 
6 = G Jo exp (~ 9 /°> d9 > 



(39) 



of light intensity. Seemingly, by a simple scheme we may have still 
gained a dB or two in performance. Because of effects such as possible 
overlap of two close pulses w(t) and even in the pulse shape of w(t) 
itself, the merits of this proposal are hard to assess without further 
study. It does appear to be an interesting possibility for a future 
detailed investigation. 

IV. INTEGRATE-AND-DUMP DETECTION— OTHER AVALANCHE 
GAIN DISTRIBUTIONS 

Personick 6 has considered the physics of a class of real avalanche 
detectors in considerable detail and has derived the following implicit 
equation for their moment-generating function M„ (s) : * 



= InM- 



1 



1 - k 



In [(1 - a)M + a], 



(40) 



* We shall drop the subscript on the mgf M„ of the gain variable when we refer to 
the particular M„ given by (40). Also, the k in this section has nothing to do with 
the k in (35) and (36). 
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where we have set 

M m M(s) = £ e'"p„. (41) 

The parameters & and a are related to the physical properties of these 
photon detectors. Since (40) has never been explicitly solved for M(s), 
we think it worthwhile to investigate the structure of M(s) implied by 
(40) in more detail. In addition to yielding structural properties of 
M (s), we shall find that (40) allows us to determine the p n of (41) 
exactly. 

To begin with, the gain G, given by G = Eg, is 



*-*-5*W 



(42) 



which, using (40), yields 



G = - r (43) 

a — k 



From (43) we see that the restrictions 

< a ^ 1 
^ k < a 



(44) 



are to be imposed on the parameters in (40). 

When o-l, (40) gives M = e>, the fir = 1 case. When k = 0, (40) 
is easily solved to give 

*C> - i - "1 ,)- ■ * = °- (45) 

Equation (45) is the mgf of the discrete geometric distribution having 
probabilities p n concentrated on the positive integers, where 

Pn = r^— (1 - a) B , n-1,2, ..-. (46) 

X CI- 

It is reasonable to treat the continuous version of this density, and 
that was done in Section II. 

In the general case of (40), the variance may be calculated to give 



var 



-»|>-<fcfl-* 



(47) 



If higher moments are desired, they can be obtained recursively from 
(40). This can be done by expanding M(s) in a power series and equat- 
ing like powers in s. 

In view of the discussion in Section III, one might prefer the de- 
tectors represented by (40) that have small variance. A simple in- 
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vestigation of (47) reveals that, for any a < I, k = uniquely gives 
minimum variance. Since even this minimum variance is large (equal 
to the mean), it may well not be a reliable guide. 

Returning to the general case represented in (40), it is evident from 
the relation 

M(s) = L e° n p n 

71 = 1 

that the mgf exists for all s ^ 0. However, it does not exist for all 
positive s, and, in fact, setting ds/dM = yields a critical value of M 
(call it M c ) given by 

and thus a critical value s c of s given by 

«.-•<*•> -fc^-rhmf, (49) 

beyond which M(s) does not exist. Note that, if b ?± (and a ^ 1), 
the value of the mgf at the critical s is finite. This shows that the far- 
tail behavior of the g variable has an exponential-like tail, with damp- 
ing factor related to s c , but in general there is a multiplicative factor, 
e.g., an inverse power that allows the mgf to be finite at its critical 
value. 

If we let s c — s = 8 > 0, M c — M(s) = A > 0, and write 



s c - s^ s(M) = s(M c - A) = s(M c ) - A ^ 



+ !**' 



2 dMl 



Mc 

+•••, (50) 



Ma 



we obtain, after evaluating the second derivative in (50), that 

A^4b^M c (51) 



or, equivalently, 



(52) 



thus exhibiting a square-root singularity of M(s) in the neighborhood 
of s c . This type of behavior is consistent with a far-tail fall-off of the 
"density" of the g variable being given by 

, exp ( — s c g) /KO , 

const. — — -; — — • (53) 
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Let us now proceed to the exact solution for the p n in (41) when 
M(s) is given by (40). We use instead z = exp (s) and write, with a 
slight abuse of notation, 

M(z) = £ 2»p„. (54) 

n=l 

Equation (40) becomes, setting M = M(z) when convenient, 

_ M 

Z ~ [M(l - a) +a] 1 /< 1 -*>" 

In (55) it is useful to make the substitutions 



(55) 



1 - k 
to obtain 

u = 



M(g) = T -^— Fl(l - a)o*'0-*>a] (56a) 

u = (1 - a)a k >«-»z (56b) 

P = — U: (56c; 



ETFW (57) 

where F(0) = and F is regarded as an implicit function of u in the 
neighborhood of u = 0. Equation (57) is a canonical form for the 
Lagrange inversion formula 7 for obtaining the coefficients Cy in the 
power series 

F = E c/u*. (58) 

The formula yields, for the present problem, 

1 (/ <* \*-« 

*"7«K 



o=4tU^)* ' I '■)" 



(59) 



or 



Ci = 1, 

i-2 

n ov>-*) rri/d - fe> + 1] . ^ 2 (60) 

From (54) and (56), the probabilities p,- are then given by 

Vi = -4- [(1 - a)a*"*-»yc,. (61) 

For (kj) large, we have, from Stirling's asymptotic formula for the 
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gamma function, 
that 



r(z + 1) ~ e- z z z+ tj2r, (62) 



^^^(kmi-k^w^*' as kj ~* x - (63) 

One can show that the behavior given in (63) is, via (61), in complete 
agreement with (49) and (53). 

Remarkably, Personick reports that Mclntyre,* from special-case 
calculations, has conjectured the exact form of (61). 

Knowing the p,- does, in principle, allow the exact calculation of the 
output statistics of the integrate-and-dump filter. The integral equa- 
tion (16), appropriately interpreted with sums, provides one such way. 
Instead of discussing this, however, we now turn our attention to 
bounding techniques. We shall make some remarks directed toward the 
Chernoff bound, used by Personick 6 for this type of problem. 

The Chernoff bound states that, if x has mgf M x (s), then the prob- 
ability that x is greater than (less than) F obeys 

Pr O > F] ^ exp (-sF)M x (s) for any s > 0. (64) 
(<) (<) 

One makes the bound as tight as possible by minimizing the right 
member of (64) over s. This, of course, assumes that M x (s) is known or 
can be obtained explicitly as a function of s. For the general class of 
avalanche diodes for which Personick derives the moment-generating 
function, we saw that s is given explicitly as a function of M and, in 
fact, an explicit function of M vs s is difficult to obtain analytically. 
Personick gets M numerically as a function of s and then proceeds to 
optimize with respect to s — a rather tedious procedure. We found from 
our experience that a simpler approach is to eliminate s in (64) by 
using (40) and then to optimize over M . This optimization still has to 
be done numerically. Nevertheless, we could generate curves very 
quickly this way. We do not present these curves here, since they do 
not reveal more than those which Personick has already published. 

For insight concerning the accuracy of the bound for present pur- 
poses, we shall apply it below to the problem of exponential gains, for 
which we have exact solutions available for comparison. 

The function appearing in the right member of (64) is, for the 
exponential gain case, 

exp (sF) exp {A[l/(1 - s) - 1]|. (65) 



* In addition to the cited reference of Personick, other experimental properties of 
avalanche photodiodes may be found in Webb, Mclntyre, and Conradi (Ref. 8). 
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Finding the optimum s is easy in this case, and (64) then yields, for 
these optimum s,' s opt = 1 - Vx/F, and, consequently, 

P[x > F > A] ^ exp [- (VX - VF) 2 ] (66) 

P[x < F < A] g exp [- (VX - <F ) 2 ]. 

From the asymptotic forms of (26) and (28), we see that the Chernoff 
bound has given us the "right exponent." 

From saddle-point considerations, this would be expected to be 
true in this problem for any M g (s) ; however, it by no means has to be 
true in general, where complex variable (saddle-point) techniques must 
be resorted to in order to decide the question. 

The optimum threshold for single-bit detection that would be ob- 
tained by equating the two expressions in (64) (for different A's, of 
course) also results in (30). Table III lists the Chernoff upper bounds 
to the bit error rate, and these should be compared to the exact answers 
shown in Table I. Numerically, the Chernoff bound is off by one to 
two orders of magnitude in error rate due to "coefficient effects." 
However, even numerically this bound is judged to perform respect- 
ably. Also shown in Table III is s opt = <*[1 - 4IjF~\, where the gain 
(a) effect has been included. For the optimum choice of F, it turns out 
that the two choices of s opt (due to two possible A's) are the negative 
of each other. Hence, only the positive one is shown in Table III. 

If one wishes to include the effects of gaussian noise here, one 
multiplies the right-hand side of (64) by the appropriate mgf, namely, 
(10). Instead of finding the optimum s for this problem, one can use 
the Sopt that held for the problem without additive noise (any s of 
appropriate sign furnishes a bound). The value a 2 = 10 4 was used in 
further Chernoff bound calculations for the M g (s) given in (41) and 
may be found in the article by Personick. 6 

V. INTERSYMBOL INTERFERENCE— INTEGRATE-AND-DUMP FILTER 

We turn now to the situation where X(0 is given by (2), i.e., a 
train of interfering pulses instead of just one of them. Personick has 
claimed that h(t) has a gaussian shape in real fibers and, hence, in 
practice only a few pulses would be expected to contribute inter- 
symbol interference. 

It is evident that, if the filter P(t) that processes the output of the 
photon detector is always positive, as, for example, for an integrate- 
and-dump filter, the presence of intersymbol interference increases 



* In setting the derivative equal to zero, one must choose the positive s that satisfies 
s < 1, since in the real-variable techniques used here, the mgf of the exponential 
does not exist f or s ^ 1 . 
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Table III — Tabulation of Chernoff bound (CB) for er ror r ate, 

exponential gain case. Also given are s opt = <*[1 — Va/F] 

for a gain 1/a = 100, and a correction exp[s' pt a 2 12] 

for a- = 10,000. The latter is a correction for 

gaussian noise. 



A« 


A,. 


CB [eq. (66)] 


s op t (gain = 100) 


exp I 


4 

4 

4 

10 

10 

10 


100 
200 
400 
100 
200 
400 


5.05 X 10~ 8 
4.16 X 10~ 17 
2.70 X 10~ 36 
1.49 X 10" 6 
1.16 X 10" 14 
2.01 X lO -32 


6.72 X 10-= 
7.54 X 10~ 3 
8.19 X 10" 3 
5.37 X 10~ 3 
6.42 X 10~ 8 
7.30 X 10-= 


1.25 
1.33 
1.40 
1.16 
1.23 
1.31 



the counts observed over any interval. Therefore, if a pulse is present, 
this intersymbol interference helps detection (helps keep output 
greater than the threshold) while, if the pulse is absent and no-counts 
is ideal, it hurts. Hence, the worst-case situation is to evaluate the 
probability of a one being decoded into a zero when no other pulses 
are present, while for the reverse error we assume all pulses are on. 
Since we are still considering an integrator, i.e., P{t) = 1, \t\ < r, 
we are still to use (6), but now for the two worst cases given we replace 
A in (6) by either 



A! = 2rX + 



j h(t)dt 



or 



(67) 



= 2rX + L ^ h(t - nT)dt. 



Of course, we assume A < A x for any reasonable operating situation. 
In addition to the threshold choice, we must also contend with the 
optimum choice of t, half the time width of the integration. This 
latter step is easily handled numerically. 

Many calculations may be done and, for the worst-case situation 
described, nothing new is involved in addition to what has already 
been discussed. As an illustration, we will deal explicitly with one ex- 
ample. We take \ = 0, no avalanche gain (g = 1), and 



h(t) = 



100 
T 



-Ml 

T J ' 



(68) 



where T is the pulse repetition rate. Thus, there is considerable over- 
lap from neighboring pulses, but not from others. Also, f h {t)dt = 100, 
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Table IV — An intersymbol interference example from Section V 



T 

T 


F 


P. [eq. (35)] 


0.1 
0.2 
0.3 
0.4 
0.5 


7 
15 
25 
34 
46 


7.3 X 10" 6 
1.5 X 10-* 
5.7 X 10"« 
1.5 X 10- B 
5.1 X 10~ B 



so the quantum limit for single-pulse detection may be read from 

Table I. 

Table IV gives the worst-case error rate for the above example, 
using formulas (35) and (38) for the Poisson case. The optimum choice 
of t here is 0.3, i.e., 30 percent toward the peak of the neighboring 
pulse. Also, a 20-percent change in the value of t does not change the 
error rate drastically. Note that we are not inferring that one should 
be careless in the choice of t, because in calculating Table IV the 
optimum threshold (F) for each t is assumed. Also, note the large 
degradation with respect to the quantum limit caused by the inter- 
symbol interference. For the present example, the error rate averaged 
over all sequences cannot be much better than shown, because the 
worst case occurs with probability I, and hence (P e ) a v cannot be more 
than a factor of 4 better. 

VI. AN INTERSYMBOL INTERFERENCE EXAMPLE AND A LOWER 
BOUND ON PERFORMANCE 

We present now a lower bound on performance which can be readily 
evaluated for the intersymbol interference problem of the last section 
[pulses given by (68)]. This lower bound is valid for optimum bit 
detection and thus sets a limit on how well any detector can do in 
coping with intersymbol interference. In particular, the bound sheds 
light on the performance in the present situation of suboptimum 
schemes such as equalization, which have found such wide application 
in voiceband telephone transmission. 

The derivation of the lower bound proceeds along lines used by 
Mazo 9 to generalize Forney's lower bound for optimum bit-by-bit 
detection in the gaussian noise. Our approach is to assume that we 
are optimally detecting the kth. bit in a sequence of (N + 1) inde- 
pendent bits, i.e., sequences of the form (2) of length (N + 1) are 
being considered. We suppose a n are binary, equiprobable, and inde- 
pendent. Let pi(x\i) and po(x\i) be the two probability densities of 
the received signal under the hypotheses o„ = 1 or 0, respectively, 
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and i denote conditioning on the iih, i = 1, • • ■ , 2 N sequence being 
transmitted. Then the probability of error for the optimum detector 
is (in somewhat formal notation) 



Pe = 2 / dxmm ^ £ pi(x\i), ^ ^Po(x|y) , 



(69) 



which, as in Ref . 9, can be lower-bounded by 



Pe^^Pe (binary i, j problem). (70) 

In (70), P e (binary i, j problem) is the probability of error which 
would result for the simple binary problem of distinguishing between 
sequence i (one having a k = +1) from sequence j (one having 
a k = 0). The bound (70) holds for all such (*, j) pairs. Finally, (70) 
holds if the sequences of length (N + 1) are shortened to N' + 1, 
with ./V being replaced by N' on the right side of (65). 

For communication in the Poisson regime, the right member of 
(70) has no known evaluation as it does for the gaussian case. What is 
known about the binary problem is the optimum detector, which is 
linear. The optimum filter P(t) and threshold F are known explicitly 
if one is deciding between equiprobable intensity functions \ a (t) and 
\ b (t). In fact, from the work of Bar-David, 10 

P(0-ta£§ (vi) 

and 

F=J\ a (t) -J\ b (t). (72) 

Thus, the set of received impulses is filtered through P(t) and the 
resulting output variable X at the end of the observation interval is 
compared to the threshold F, choosing \ a (t) if X > F and \b(t) other- 
wise. Assuming \ a (t) is transmitted, the moment-generating function 
of X is, from (5) and (71) (recall g — 1 in this section), 

M x (s) = exp ["jxoWCexp {«ln OiWAtWH) " 1]*] 

= exp I" |[Xi-(0Xi'(0 - X«(0]<al • (73) 

From this mgf, one can see why the right side of (70) is not known in 
general. 

We now apply (73) to the intersymbol interference of the previous 
section, where h{t) is given by (68). We choose N = 2, \i(t) to cor- 
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respond to the pulse sequence (1, 1, 1) and \b(t) to correspond to the 
pulse sequence (1, 0, 1). When applied to (70), we interpret the results 
as applying to the center bit of the sequence. We have, explicitly,* 

\i(t) = 1 for \t\ ^ 1 

X (0 = \t\, for \t\ ^ 1 (74) 

Xi(0 = Xo(0 for \t\ > 1. 

Since, from (71), P(t) = for \t\ > 1, the detection interval 
t G [ — 1, !]• Using (74) in (73), the decision variable has mgf 

M x (s) = exp^j^- s - lj- (75) 

Remarkably enough, this is the moment-generating function of the 
random variable dealt with in Section II ; in the notation of that sec- 
tion, it corresponds to A = 1, a = 2. The density is given by (20), 
and the threshold is, from (72) and (74), to be set equal to unity. 
Putting this all together, (70) becomes 

P e ^ t e- 1 f a J- e-t'Ii (2V2x)dz. (76) 

Or, scaling (76) to reinsert the factor of 100 in front of (68), 

P e >\ e~ m I" J— e- 2x l!(2^mkc)dx. (77) 

4 ./ioo V x 

So an excellent approximation in the right-hand side of (77) may be 
evaluated via (26) to give 

P e ^ |(i)*Q(V400 - V200) W 5.06 X 10- 10 . (78) 

The numerical value of (78) should be compared with Table IV for 
performance with integrate-and-dump filter and Table I for the 
quantum limit. Indeed, for this case our bound shows that the optimum 
detector performance is still far from the quantum limit and, in fact, 
is roughly only 2.2 dB (comparing powers of 10) better than the 
integrate-and-dump filter. 1 The present problem seems to imply that 
equalization,* in particular, cannot be expected to approach the 
quantum limit bound for the type of distortion found in present optical 
fibers. In fact, a simple integrate-and-dump receiver with properly 



* For the moment, we ignore the factor of 100 in (68) and also set T = 1. These 
are reintroduced only in the final numerical calculations. 

f More precisely, the figure is 2.9 dB for strong signals. 

* Some references on equalization for optical communication systems are Refs. 1 
and 11. 
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chosen threshold compares well with a lower performance bound. The 
above problem ignored many practical factors, but in fact ignoring 
them focused even more on the pure intersymbol interference problem 
in the Poisson regime. It would seem that effects such as dark current 
and finite width of w(t) would surely make the integrate-and-dump 
and the optimum detector perform even more equally, and it would 
seem too much for an equalizer to compensate for gain jitter, which is a 
rather nonlinear effect. 

Another linear filter P{t), which performs better than the integrate- 
and-dump, may be inferred from (71). This is discussed and evaluated 
in the appendix for the present problem. This new linear filter has a 
worst-case exponent approximately 1 dB better than the integrate- 
and-dump situation. 

APPENDIX 

A New Filter 

We have already noted that (asymptotically) the integrate-and- 
dump filter performs within 2.9 dB of a lower bound on performance 
for the optimum processor for our particular example. We now show 
how a modified P(t) can perform within 2 dB of this bound. We 
confine ourselves to the worst case again, for which, we recall, the 
best integrator had P(t) = 1 for | < | ^0.3 (choosing T = 1). The worst 
case with signal present was Xi(<) = 1 — \t\, \t\ < 1, and X o (0 = \t\, 
\t\ < 1, for the worst case with signal absent. Now the optimum filter 

P(t) ■In 1 T.J* 1 , |*| < 1, (79) 

I I 

which distinguishes between these two signals, is not always positive 
(it is negative for \t\ > §). Therefore, if (79) were used, there could 
be no claim for a worst-case bound. However, we modify (79) and use 

P(t) = ]n^M, \t\ <\ (80) 

instead. The filter represented by (80) is always positive, and therefore 
worst-case claims still obtain. The filter (80) clearly has to outperform 
our integrate-and-dump one, since the latter integrated only to 
\t\ = 0.3, while (80) is optimum for an observation interval \t\ ^ 0.5. 
The optimum threshold for (80) is, from (72), 

F = / \i(t)dt - j ^ \ Q (t)dt = i (81) 

Using the Chernoff bound for the case when \ (t) is sent, we have, from 
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(64) and (73), 

P e ^ exp J y\o(0[> ,P(,> - 1] " sF \ 

= exp J2 J* ^"-(l - tYdt - i - I , s > 0, (82) 

where we have used the expression for Xo(0> tne n l ter ( 80 )> anc * thresh- 
old (81). If we let u =■ (1 — OAi then we may write 

£ '"*-**-£ try*- (83) 

Two integrations by parts give 

A (HMO 5 ** = 8 + 4 + —2— h !+**> (84) 

or, using (84) in (82), 

P e ^ exp [t (i - 1) f" j^ dn] • (85) 

Equation (85) makes it evident that the exponent in (82) will be 
negative for < s < 1. If we expand the 1/(1 + u) part of the inte- 
grand in (80) in powers of (1/w) and integrate term by term, the 
exponent in (85) becomes 

( — l) k 

8(8-1) E V ' 



i f^ k + 2 — s 



= s(s — 1) E Ti — TTi w? — i~~o T\' (^6) 

k=o (k + 2 - s)(k + 3 - s) 

k even 



Convergence in (86) can be improved if we write 



E = * E +\ E -* E 

k even all J: k even A odd 



and use the fact that 

E 



1 (x + n) {x + n + 1) 1 + x 
to obtain 

S(S_1) [2(2-s) + *?o (fc+2-s)(A;+3-s)(A;+4-s) J 



(87) 



t even 



The optimum s is easily found numerically by plotting (87); we 
truncated the sum after k = 10. We find the optimum s is about 0.6, 
giving a value of (87) of 0.11138. As a check on the possible accuracy 
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of our use of (87), we note that our technique gives 0.10696 when 
s = \, for which the exact answer can be shown to be t/8 — \ tt 
0.10730. Thus, the Chernoff bound is 

Pe ^ exp (-O.lllAo) 

rx (88) 

A = / \o(t)dt, 

while (73) yields as a lower bound something which behaves ex- 
ponentially as 



exp 



(V * - ^ )2 1) = exp (-0.172Ao). (89) 



The exponent of (88) is 1.9 dB worse than that of (89). Concluding, we 
note that (80) has a logarithm singularity at t = 0. Including dark 
current in the X,(£) will remove this, and will also decrease the im- 
provement which this kind of filter provides over the integrate-and- 
dump filter. 
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